Retrieval of Research-level Mathematical Information Needs: A Test Collection and Technical Terminology Experiment

In this paper, we present a test collection for mathematical information retrieval composed of real-life, researchlevel mathematical information needs. Topics and relevance judgements have been procured from the on-line collaboration website MathOverflow by delegating domain-specific decisions to experts on-line. With our test collection, we construct a baseline using Lucene’s vectorspace model implementation and conduct an experiment to investigate how prior extraction of technical terms from mathematical text can affect retrieval efficiency. We show that by boosting the importance of technical terms, statistically significant improvements in retrieval performance can be obtained over the baseline.